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ABSTRACT 



The three-point sextant fix has been used for the 
horizontal positioning of vessels in nearshore hydrographic 
surveys since 1775 • However, this method has only recently 
been modeled mathematically to quantify the effects of 
various errors on fix accuracy. Positioning error in the 
three-point fix depends on the magnitude of the random and 
systematic errors in the angle measurements and the fix 
geometry. Random errors in horizontal sextant measurements 
were investigated by analyzing over 1400 angular observa- 
tions, both at sea and on land. These errors were found to 
vary with the clarity of the signals being observed, the 
stability of the vessel and the experience of the observer. 
The upper and lower bounds for one standard deviation 
were found to be about 2.7 and 0.6 minutes of arc respec- 
tively. In addition, systematic errors resulting from 
angular differences due to the direction of rotation of the 
micrometer drum were examined as well as the variability in 
the determination of sextant index error. 
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I. INTRODUCTION 



A . HISTORICAL BACKGROUND 

Instruments for measuring the altitude of the sun or 
stars have existed for over 2,000 years, hut it was not 
until 1730 that the forerunner of the modern marine sextant 
appeared. Two inventors, John Hadley of England and Thomas 
Godfrey of Philadelphia! simultaneously designed instruments 
using one fixed mirror and one movable mirror to measure 
angles [May 1963 ]• Both instruments allowed the movable 
mirror to rotate through an arc of 45° . but due to their 
double mirror construction were able to measure angles up to 
90°.^ Hence, they were called quadrants, although octants 
would have been a more proper name. These quadrants were 
not readily adopted by navigators and it was 1?50 before 
Hadley's quadrant was in general use aboard vessels of the 
East India Company [Cotter 1972] . 

The need to measure angles greater than 90° prompted 
Captain Campbell R.N. in 1757 to suggest the enlargement of 
the arc of Hadley’s quadrant to 60° enabling the measurement 
of angles up to 120°. Hence, the name sextant from the 
Latin word sextans . "the sixth part" [Bowditch 1977] • 

^ Bowditch (1977) discusses the optics of this 
construction which is similar to the modern sextant. 
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Although the sextant was designed primarily for 
measuring vertical angles, it can be held on its side to 
measure horizontal angles as well. Rev. John Mitchell first 
suggested the use of the sextant for "measuring horizontal 
angles in hydrographic surveying in February I 765 [Cotter 
1972 ]. Mitchell's method of fixing a vessel consisted of 
intersecting the line of position derived from a horizontal 
angle between two known points^ and a position line obtained 
from a compass bearing to one of the points. In l77i » the 
first Hydrographer to the Admiralty, Alexander Dalrymple, 
suggested determining position by intersecting the lines of 
position derived from horizontal angles between three or 
more known points [Cotter 1972]. This principle had been 
known by land surveyors since the early seventeenth century 
but its application aboard ship was delayed due to the lack 
of an accurate angle -measuring device . 

The first application of Dalrymple 's suggestion was made 
by Murdoch Mackenzie II. He surveyed the channels off the 
Kent coast of England between 177^ and 1777 > Positions 
could be plotted quickly by using a device called a three- 
arm protractor or station pointer [ Admiralty Manual of 
Hydrographic Surveying 1965]- This method for positioning 

p 

A known point is a reference station whose geodetic 
coordinates have been determined. Also called a horizontal 
control point. 
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is called the three-point fix or resection method and is 
used today worldwide for inshore hydrographic surveys . 

There have been several changes in the equipment used 
in the three-point fix method over the last 200 years. An 
endless tangent screw for continuous tracking and a micro- 
meter drum for increased accuracy were added to the sextant 
during the twentieth century. The marine sextant is fitted 
with darkened shade glasses for observation of the sun. A 
specialized sounding sextant has been developed specifically 
for hydrographic surveying using lighter weight materials 
and more rugged construction [Ingham 1975 ] • In addition, 
the sounding sextant has a wide angle low magnification 
telescope and a micrometer graduated in minutes of arc. In 
recent years an electronic digital sounding sextant was 
developed to enter observed angles directly into a mini- 
computer aboard a vessel [Umbach 1976] . Despite these 
improvements in the sextant the most draipatic equipment 
change has been in the area of position plotting. Shipboard 
computers and automated plotters have largely replaced the 
three-arm protractor resulting in increased accuracy. 

B. THREE-POINT FIX METHOD AND POSITIONING ACCURACY 

Figure 1 illustrates the geometry of a three-point fix. 
The known stations are depicted by triangles and labelled A, 
B and C. The vessel is located at point P and the observed 
sextant angles are given by the symbols 0i and 02- The 
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Figure 1 . Three-Point Fix Geometry 



angle ©i and the points A and B define a locus of points 
which is a circle through A, B and P. At any point on that 
position circle, or line of position, the angle between A 
and B is always Similarly, there exists a position 

circle for points B and C and angle 02* A third line of 
position through points A and C is defined utilizing the 
angle ( ©i + ©2)* 

As can be seen in Figure 1 , the three lines of position 
intersect at point P. The accuracy of the position of P 
depends on the errors in the measured quantities used for 
position determination. Errors are classified into three 
types: blunders, systematic errors and random errors. 

Blunders are simply mistakes such as misreading a sextant 
or improperly identifying a signal. ^ They are eliminated 
from the data by compairison with redundant or related 
observations and careful editing. Systematic errors follow 
some mathematical or physical laws and therefore have a 
fixed relation to a set of conditions. For example, 
elevation differences in observed signals cause systematic 
errors in sextant measurements. In this case the conditions 
(the elevation differences) can be measured and the correc- 
tions can be calculated and applied. However, the laws 
associated with systematic errors are not always known. 

3 A signal is a natural or artificial object located at 
a survey station (known point) which is used as a sighting 
point for sextant measurements. 
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Systematic errors from unknovm sources can be minimized by 
following sound measuring techniques and instrument 
calibration procedures. Random errors are inherent in all 
physical measurements and can not be removed from the data. 
Their effects must be estimated statistically. An example 
of random errors would be measuring a known 50° angle 
several times and obtaining sextant readings between 
49°57.2’ and 50°02.8' . 

Blunders and systematic errors affecting the accuracy 
of positions determined by the three-point fix method are 
discussed in Appendix A. The random errors in horizontal 
sextant measurements at sea are the subject of this paper. 

C . OBJECTIVES 

Until recently, there have been very few attempts to 
quantify the accuracy of sextant positions. Several 
formulas have been developed in the last decade to do this 
using the magnitude of the random errors as one of the 
parameters. Tozzi (1974) developed a series of equations 
which included among other variables, the standard deviation 
of the random error in measuring angles. Thomson (1977) 
used the confidence intervals associated with the two angle 
measurements of a three-point fix. The equation in 
Bowditch (1977) includes the error in measurement of the 
horizontal angles. Heinzen (1977) used the mathematical 
notation for the standard deviation of the observed angles 
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in his development. Dedrick (1978) developed a formula for 
the area of the error ellipse about some position given the 
standard deviation of the sextant angle measurement error. 

There seems to be no agreement among the various authors 
as to the magnitude of these random errors. The table below 
illustrates this. 



Author 


Year 


Error 


Goodwin 


1973 


6' 05.5" 


Ingham 


1974 


+ 0.5' 


Ingham 


1975 


+ 1.0' 


Thomson 


1977 


4' 


U.S. Coast Guard 


1978 


several minutes 


Dedrick 


1978 


few to several 
minutes 


Bodnar 


1978 


0.7’ - 1.3' 



TABLE I. Previous Values for the Magnitude of Random' 
Errors 



The errors mentioned by Goodwin, Thomson, Dedrick and 
Bodnar specifically refer to the standard deviation of the 
random errors while Ingham and the U.S. Coast Guard do not. 
It should also be noted that Tozzi (197^) used Goodwin's 
standard deviation in his development and Heinzen (1978) 
used Ingham's 197^ value. All of the above errors are 
estimates arrived at through experience except those of 
Goodwin and Bodnar. Goodwin arrived at his result by 
calculating the standard deviation of 32 angular measure- 
ments between two well defined stations. He had 32 
experienced navigating officers each measure the angle once. 
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Bodnar's figures resulted from a one day experiment 
aboard the NOAA Ship DAVIDSON while moored at a pier at 
Lake Union in Seattle, Washington. Theodolite observations 
were made from the ship to three easily identified objects 
which were generally at the same elevation. Conditions of 
extreme vessel stability due to tight mooring lines, little 
wind and no tide enabled Bodnar to achieve agreement between 
successive theodolite observations of about 0.5'* The two 
angles (one of about 6° and the other about 50°) were then 
measured 30 times each by six officers. The means and 
standard deviations were calculated for each of the 12 data 
sets. All of the means were within 0.5' of the angles 
determined by the theodolite measurements. The individual 
standard deviations ranged from 0.7' to 1.5'* No cumula- 
tive statistics were determined. 

Dedrick (1978) attempted to use historical data to 
arrive at his estimate. During a survey of south San 
Francisco Bay between 1857 and 1858 it was common practice 
to take full rounds of angles at a station. That is, angles 
were measured between objects all around the horizon. He 
studied 17 rounds of angles each consisting of four to 
seven sextant angles. The disagreement between each round 
and 380 ° was generally between 10' and 20' of arc, but 
ranged from 2' to 55' of arc. He states that "this data 
would suggest that as an extreme upper limit, values of 
of a few to several minutes of arc might be appropriate." 
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The main objective of this study was to quantify the 
random errors in sextant angle measurements under varying 
conditions. The factors upon which these errors are depen- 
dent are: (1) the ability of the sextant observer, 

(2) the visibility and distinctness of the signals, and 

(3) the stability of the platform. The ability of the 

sextant observer was analyzed by comparing more experienced 
observers with those with less experience. All observations 
were made across water on clear, somewhat windy days so that 
horizontal refraction was at a minimum. The distinctness of 
the signals was altered by using telescopes on the sextants 
for some observations and not for others . Platform sta- 
bility was analyzed by measuring angles under three 
conditions: (1) vessel in moderate to heavy seas, 

(2) vessel in calm seas, and (3) observer at a stationary 
point on a wharf. 

During the course of data collection a few other 
questions arose which were related to random errors. Is 
there a difference in angle measurement if the micrometer is 
turned clockwise or counterclockwise? How much do individ- 
uals vary when determining index error on the same sextant? 
Are the manufacturer's stated instrument errors correct? 
Answers to these questions and the analysis of the random 
errors are discussed in later sections . 

The scope of this paper does not include the positional 
accuracy of three-point sextant fixes. Tozzi (197^) » 
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Heinzen (1978) aind especially Dedrick (1978) covered this 
subject in detail. No attempt was made to evaluate the 
random errors while a vessel was underway as it would be 
when running a hydrographic survey line . The data analysis 
in this thesis corresponds to the use of the three-point fix 
for calibrating or evaluating electronic positioning control. 
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II . DATA COLLECTION METHODS 



Raw data for this study consists of sextant angle 
measurements observed in the southern portion of Monterey 
Bay, California. This area and the horizontal control 
stations used during the project are shown in Figure 2. 

All of the station positions were determined by third-order 
methods or better by personnel of the National Ocean Survey. 
These stations, their positions, elevations and station 
numbers and the sources of this information are shown in 
Table II. The data was collected both at sea and on land 
and will therefore be discussed separately. All sextants 
were checked each day for adjustable errors and were found 
to be satisfactory. 

A. CRUISE DATA 

The 126-foot long research vessel ACANIA was used as 
the observation platform for the anglemen (sextant 
observers) during the two data collection cruises. The 
position of the ship was determined for every sextant 
observation by the standard theodolite intersection method 
described by Umbach (1976). Briefly, this method consists 
of occupying two horizontal control stations with 
theodolites. Each theodolite operator measures the angle 
between another known point and the object to be located. 





CRUISE I 
CRUISE H 



Figure 2. Project Area and Station Locations 



20 



STATION ELEVATION 

NUr/lBER STATION NAME LATITUDE (N) LONGITUDE (W) ( METERS ) SOURCE * 
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TABLE II - List of Stations 



Since the azimuth can he computed between any two known 
points, the azimuth to the unknown point can be determined. 
Therefore, the theodolite measurements from each station 
produce lines of position, the intersection of which is the 
location of the unknown point. To ensure the correctness 
of the beginning azimuth, a check azimuth to a second known 
point is usually observed. 

Stations MUSSEL (10?) and MONTEREY CO. DISK (3OI) were 
occupied with Wild T-2 theodolites on both cruises. USE 
MON (ilO) was observed to obtain the initial for MUSSEL 
(107) and MONTEREY HARBOR LT 6 (205) was the azimuth check. 
The theodolite observer at MONTEREY CO. DISK (30^) 
initialled on MUSSEL (10?) and sighted on USE MON (110) as 
the azimuth check. (Subsequent references to these and 
other stations will only be by the station numbers indicated 
in Table II.) Both theodolite operators attempted to use the 
center of the three anglemen as a target. However, since 
this was not easily visible at all times, a bright orange 
float, 2 feet in diameter, was tied to the ship's rail 
beside the observers. 

The sextant observers consisted of six individuals with 
three levels of experience in measuring angles with sex- 
tants. Two anglemen (1 and 2) had extensive horizontal 
angle measurement experience. Two others (3 and 4) had 
extensive experience with measuring vertical angles with a 
sextant, but none with measuring horizontal angles. The 
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final two (5 and 6) had no prior sextant experience. The 
six observers were divided into two groups of three . The 
more experienced group consisted of the first two observers 
and anglemen number 3* Each group of three measured angles 
simultaneously from the aft portion of the upper deck of 
ACANIA. The observers stayed within one meter of each other 
to minimize the effects of eccentricity. 

Observations from the ship were coordinated with those 
ashore by using three portable CB radios. To reduce the 
positioning errors caused by theodolite mispointings the 
captain of ACANIA attempted to stay within a small area 
about 450 meters NNW of the Coast Guard pier. This position 
minimized the distance to the two theodolite locations and 
gave an intersection angle of near 90°. Unfortunately, 
weather conditions made it impossible to maintain station. 
Figure 2 shows the areas of operations for both cruises. 

Cruise I took place under adverse weather conditions. 
Winds were 20-25 knots from the WNW with seas of 
approximately 5 to 8 feet. ACANIA, with its 126-foot length 
and 22-foot beam, provided a very unstable platform. By 
not using scopes on the sextants under these conditions of 
poor platform stability, the largest value for the standard 
deviation of the random errors was obtained. 

Each angleman measured the angle between signals 110 and 
205 a total of 32 times. In addition, the more experienced 
group measured the angles required to compute ten pairs of 
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three-point fixes. Each pair of fixes was derived from a 
left angle 02(110/205) » a right angle ©2(205/202) and a 
right check angle ©^ (205/107)* The fix determined from 
angles ©2 and ©2 was designated the main fix and the fix 
computed from ©2 and ©^ was called the check fix. The 
distance from the main fix to its corresponding check fix 
was determined and is called the inverse distance . Blunders 
in the horizontal angles can be detected by analyzing the 
size of these inverse distances. If the values are less 
than five meters, then the National Ocean Survey considers 
that no blunders have been made. For the ten pairs of fixes 
from Cruise I the mean inverse distance was 1.218 meters 
with values ranging from 0.003 to 2.390 meters (standard 
deviation = 0.80 meters). Planned operations to collect 
additional angle data for three-point fixes were cancelled 
due to worsening weather conditions and failing radios. A 
total of 222 sextant angles were collected for analysis. 

The weather conditions for Cruise II were much better 
than those for Cruise I . The wind was from the southwest 
at 5 to 10 knots with no appreciable seas. However, long 
period swells of 1 to 2 feet caused some vessel motion. 
Scopes were used on the sextants in an attempt to determine 
a reasonable least value for the standard deviation of the 
random errors at sea. It was planned to also collect data 
with no scopes, but a heavy rainstorm reduced visibility to 
less than 1,000 meters. 
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On this cruise each angleman took measurements of 
the angle 110/205* As in Cruise I, angles were observed to 
compute pairs of three-point fixes. The same signals were 
used for both cruises. Each group measured a total of 96 
angles resulting in 32 pairs of fixes. The mean inverse 
distance was 0.7^9 meters with values ranging from 0.011 
to 2.555 meters (standard deviation = 0.61 meters). The 
total number of angles measured for Cruise II was 38 ^* 

B . STATIONARY DATA 

All of the data collected at sea was subject to possible 
errors from the observers not being in the exact same 
location and from the theodolite intersection method. These 
errors are discussed in Appendices A and B respectively. 

The random errors were also influenced by platform 
stability. To evaluate the effects of this factor the 
observers measured a series of angles from station 350 on 
Wharf Number 2 at the Monterey Harbor. A T-2 theodolite was 
used at 350 to measure the horizontal angles between llO and 
206 and between 110 and 302. These angles were measured 
according to the third-order specifications prescribed by 
Umbach (1976) which require four measurements with different 
plate settings, all within 5 seconds of the mean. 

Each of the 6 anglemen stood directly above 350 and 
measured both angles 30 times with scopes and 30 times 
without scopes. In addition, each 30 observation set was 
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divided such that 15 angles were measured with the 
micrometer drum being turned clockwise or decreasing in 
value and 15 angles were measured with counterclockwise 
motion. Since the observed stations were not at the same 
elevation as 350 , the observed angles were not horizontal 
angles. The equation from Umbach (1976) for converting 
inclined angles to horizontal angles and vice versa is 
given in Appendix A. The angular elevations of each object 
at 350 were measured with a T-2 theodolite. Hence, the 
horizontal angles 206/110 and IIO/302 were converted to the 
inclined angles that were observed by the sextants. The 
errors were then calculated for each of the 720 sextant 
angles . 

C. INDEX ERROR AND INSTRUMENT ERROR DATA 

Tests were conducted to determine if any systematic 
errors were unaccounted for in the previous data. Each 
individual determined index error for his sextant every 
time he made a set of observations. However, there were 
larger than expected differences of index error between 
individuals using the same sextant. Individual determina- 
tions also varied from day to day. Therefore, index errors 
were studied by having each angleman make 30 measurements 
of index error with each of the three sextants. The 
procedure consisted of holding the sextant vertically, 
observing the sea horizon and bringing the direct and 
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reflected images into coincidence. As before, 15 
measurements were made with a clockwise micrometer drum 
movement and 15 with a counterclockwise movement. A total 
of 5^0 index error measurements were analyzed. 

Instrument error was also analyzed for each sextant. 
Recall that instrument error consists of graduation error 
and centering error. This is determined for each instru- 
ment by the manufacturer and the results are posted inside 
the sextant case. Each of the three sextants used on this 
project had posted correctors of 0.0 minutes for each 15 ° 
increment of arc. To check some of these values a method 
suggested by CDR. James Wintermyre of the NOAA Pacific 
Marine Center's Operation Division in Seattle, Washington, 
was used. He reasoned that to properly "calibrate" a 
sextant or determine its instrument error for various 
angles, accurate theodolite angles must first be observed 
between objects. The sextant could then be used to measure 
the same angles. To maximize the accuracy of this • calibra- 
tion the sextant must be perfectly horizontal and over the 
identical point from which the theodolite angles had been 
measured. To accomplish this he designed a sextant template 
that would mount in a T-2 theodolite tri-brac and would 
accommodate three different makes of sextants. The template, 
tri-brac and a sextant are shown in Figures 3 4. 

Initially a temporary reference station was established 
from which several objects at the same elevation could be 
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Figure 3* Template, Sextant and Tribrac - Disassembled 




Figure 4. Template, Sextant and Tribrac - Assembled 
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sighted on. The objects were the southwest edge of a 
chimney, a flagpole on Wharf Number 2 and the southern edge 
of the AMERICAN GAN CO. STACK (202). Their elevations were 
determined to be the same as the theodolite by setting 90 ° 
on the vertical circle. The angles between the objects 
were measured with the theodolite, again to third-order 
specifications. Unfortunately, the theodolite did not 
have a removable tri-brac . Hence the template was mounted 
on another tri-brac which was centered over the station. 

The "bull's-eye" level bubble of this new tri-brac was out 
of adjustment. But since the proper vertical location on 
each target was well determined by the theodolite, the 
tri-brac was adjusted to allow sighting on these points 
with the sextant. Each angle was measured 30 times with 
each sextant - 15 times with a clockwise micrometer movement 
and 15 times in a counterclockwise direction. Thus, a total 
of 180 angles were observed for analysis. 
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III. DATA REDUCTION PROCEDURES 



All computer work for this project was done on the 
IBM 360/67 system at the Naval Postgraduate School's W.R. 
Church Computer Center. All programs were written in 
FORTRAN IV. Program UCOMPS was used to determine positions 
from three-point sextant fixes and T -2 theodolite inter- 
sections . It was provided by the National Ocean Survey - 
NOAA, Rockville, Maryland. Program INVERS was utilized to 
compute the lengths and azimuths of lines between known 
points. This program was obtained from the National 
Geodetic Survey - NOAA, Rockville, Maryland. Library 
routine HISTF was used for the statistical analysis of all 
data for this project. 

Error analysis requires that the best estimate of a 
measured quantity be determined. Usually this value is the 
mean of a number of measurements of a particular angle or 
distance. However, the sextant angles observed during the 
two cruises could not be treated in this manner since ACANIA 
was always moving. Thus each angular measurement was a 
unique and unrepeatable observation. Therefore, the best 
estimate of each sextant angle observed aboard ACANIA was 
derived utilizing the theodolite determined ship positions. 
Inverses were computed from each position to the horizontal 
control stations upon which the sextant observations were 
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made (station numbers 110, 205. 202 and 107)- The 

difference in forward azimuth to any two stations from the 

ship's position was chosen the best estimate of that angle. 

Likewise, the best estimates for the sextant angles observed 

at Wharf Number 2 were the inclined angles derived from the 

T-2 theodolite measurements there (see page 26). 

Errors ( e ) were calculated for each observed sextant 

angle (x) by subtracting the best estimate of the angle (X) . 

This is expressed mathematically as ^ i ~ ^i ~ X where 

i is the number of the observation. The mean ( e ) of the 

L 

errors for each data set was then calculated as follows: 
n 

e=^. Z^e^./n where n is the total number of observations. If 
the errors in the sextant angles are normally distributed 
and totally random, then by definition e must be equal or 
nearly equal to zero. If e does not equal zero for each 
data set, then sampling errors and/or undetermined 
systematic errors were present. The bounds of sampling 
errors for a given probability are directly proportional to 
the sample standard deviation of the data set and inversely 
proportional to the square root of the number of errors 
analyzed. 

The sampling and systematic errors must be eliminated 
when calculating the sample standard deviation ( a ) of the 

^ The errors determined for each observer under each 
set of conditions are referred to as a data set. 
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random errors. This was done for each data set by using 



/\ 

the equation o 




-e)^]/(n-l) 



It can be shown that a is a measure of precision and i is 
a measure of accuracy. 

The mean of the errors ( i ) and the sample deviation 
( a ) for each data set and for some selected combination 
of data sets were calculated. These results are discussed 
in the next chapter and are summarized in Appendix C . 

Graphs of the distribution of the errors for some of the 
combined data sets are also shown. These graphs called 
frequency polygons are formed by connecting the mid-points 
of the tops of the bars in the histograms of the data. 

A coding system was devised to simplify references to 
various data sets or data set combinations. The code 
consists of four or five characters and is shown in 
Table III. 

The fifth character of the code is used for the cruise 
data if angle was reobserved when collecting data for 
three-point fixes. Hence, data set INI - ©i(2) would refer 
to the second data set collected by observer 1 while mea- 
suring angle ©j^ with no scope on Cruise I. Combinations 
of data sets are referred to by the part of the data that 
is common to all sets. For example, combination data set 
IN - would refer to all no scope observations of angle 
©1 on Cruise I. 
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(1) Data Origin: 

I - Cruise I 

II - Cruise II 

W - Wharf Number 2 (Stationary) 

B - Beach Lab (Index Error Data) 

(2) Use of Scopes on Sextants: 

N - no scopes 

S - with scopes 

(3) Observer Number (experience level decreases as 

this number increases: 

1 through 6 

(4) Angle Designator: 

©1 - 110/205 

®2 - 205/202 
©3 - 205/107 
©4 - 206/110 
©3 - 110/302 

(5) Repeated Data Set (if necessary): 

(1) first set 

(2) second set 



TABLE III. Data Set Reference Code 
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IV . RESEARCH RESULTS 



The factors affecting the random errors in horizontal 
sextant measurements are the ability of the sextant 
observer, visibility and distinctness of the signals and 
the stability of the platform. The effects of these factors 
are not independent, and it is therefore impossible to 
isolate the contribution of each error source. In addition, 
errors induced by the theodolite intersection method and 
errors due to eccentricity of the observers are included in 
all of the data collected at sea. Statistics for all data 
sets are summarized in Appendix C. 

A. ABILITY OF THE OBSERVER 

This factor is best evaluated by comparing the results 
of the data sets for each observer under similar conditions. 
The mean of the errors and the sample standard deviation are 
denoted by i and a respectively. Cruise I data is 
denoted by data sets beginning with IN in Table C-1 . The e 
for observer six seems extremely large . This was likely 
due to the seasickness that the individual experienced while 
measuring angles. As mentioned before, the sea conditions 
for Cruise I were extremely rough. 

The data from Cruise II is shown in Table C-1 beginning 
with code IIS. The strong positive bias of the e's will be 
discussed in section F. 
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The results of 24 data sets taken at Wharf Number 2 
are indicated in Tables C-2 and C-3 by the codes beginning 
with WN and WS . These data sets show no major differences 
between the experienced and inexperienced observers. 

Observer experience was also evaluated by comparing 
simultaneously observed sextant angles. This eliminated 
the effects of positioning error from the theodolite inter- 
sections. The mean of the differences for each group is 
shown below in Table IV for both cruises. Group 1 consisted 
of observers 1-3 and group 2 of observers 4-6. Two angle 
differences from group 1, Cruise I were rejected due to an 
obvious blunder in one angle of lO'. All angle differences 
involving observer 6, Cruise I were rejected due to their 
unreliability. The values in parentheses indicate the 
number of differences used to compute each mean. Some 
error was introduced due to the eccentricity of the three 
observers in each group. However, this error was assumed 
to be the same for each group. 



Cruise I 



Cruise II 



Group 1 
Group 2 



2.54 (19) 
3.10 (32) 



1.29 (96) 

1.66 (96) 



TABLE IV. Mean Differences Between Simultaneously 
Measured Angles (Minutes of Arc) . 
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The agreement of the group 1 observers was 0 . 3 ' to 0.5' 
better than that of the group 2 observers. This was the 
best indication of improvement in measuring horizontal 
sextant angles due to increased experience. 

B. DISTINCTNESS OF SIGNALS 

The effect of signal clarity or distinctness was 
determined by comparing data collected without scopes 
(Cruise I) on the sextants to that collected with scopes 
(Cruise II). The data for (angle 110/205) from Cruises 
I and II are denoted by IN- 0^ and IIS- 0 ^ . The cumula- 
tive data from these data sets is shown graphically in 
Figure 5* This illustrates the increased dispersion of 
the Cruise I data compared to that of Cruise II. 

•As a further comparison, inverse distances were 
determined between the main three-point sextant fixes and 
the corresponding check fixes. The results are shown 
below in Table V. Again the number of observations used in 
computing the statistics are shown in parentheses. 

Cruise I (10) II (64) 

i • 1.22 0.75 

o 0.80 0.61 

TABLE V. Statistics for Inverse Distances Between Main 
Three-Point Fixes and Check Fixes (Minutes 
of Arc ) . 
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ERROR (MINUTES) 




ERROR (MINUTES) 



Figure 5 . Data Distribution for ©i (Angle 110/205) - 
Cruise I (top) vs. Cruise II (bottom) 
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V 






The decreased dispersion in the data from Cruise II 
versus that from Cruise I is due not only to the use of the 
scopes on the sextants, but also to the decrease in vessel 
motion. The data at Wharf Number 2 eliminated vessel 
motion completely. There were dramatic decreases in the 
values of e and a when scopes were added to the sextants. 
The cumulative statistics of the data are illustrated in 
Figure 6. Hence, the use of scopes on sextants decreases 
the magnitude of random errors in horizontal sextant angle 
measurement . 

C. PLATFORM STABILITY 

The effects of platform stability on random errors are 
best evaluated by comparing the Cruise I data to the data 
collected on Wharf Number 2 without scopes and comparing 
the Cruise II data to the data collected with scopes on 
Wharf Number 2 . These four sets of data are shown in 



Figures 5 and 6 . 


The statistics 


are summarized 


below in 


Table VI. 










Wharf 2 




Wharf 2 


Cruise I 


(No Scopes) 


Cruise II 


(Scopes) 


i -0.32 


+0.89 


+ 1 . 63 


+0.20 


S 2.69 


2.07 


1.58 


0.64 



TABLE VI. Comparison of Cruise Data to Wharf Number 2 
Data (Minutes of Arc) . 
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4 

I li 



FREQUENCY FREQUENCY 




-10.0 -5.0 0.0 5.0 10.0 

ERROR (MINUTES) 




ERROR (MINUTES) 



Figure 6 . Distribution of Wharf Number 2- Data - Without 
Scopes (top) vs. With Scopes (bottom) 
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The increased dispersion of the cruise data is apparent 
in the figures as well as the table. Hence, when vessel 
stability is increased there is a significant reduction in 
the magnitude of random errors. 

D. DIFFERENCES DUE TO DIRECTION OF ROTATION OF THE 

MICROIVIETER DRUM 

The differences in the Wharf Number 2 data created by 
clockwise rotation of the micrometer drum versus counter- 
clockwise motion are shown in Tables C-2 and C-3* 
Observations 1-15 were made with clockwise micrometer drum 
movement and correspond to decreasing values. Counter- 
clockwise rotation was used for observations 16-30. The 
magnitude of the differences in sample standard deviations 
is small - none are greater than 0 . 20 ' for the data taken 
with scopes and only 3 greater than 0.40' for the no 
scope data. Observers 1 , 3 and 5 seem to measure smaller 
angles when using counterclockwise micrometer rotation. 
These three all have negative i values with average Ac's 
of -0.68', -0.60' and -O. 5 O', respectively. The other 
observers did not exhibit this tendency as strongly. 

The procedure for collecting index error information 
at the Beach Lab property was described on page 27 . The 
corresponding index correction (the negative of index 
errors) are summarized in Table C-4. Scopes were used on 
the sextants. As before, observations 1-15 were made with 
clockwise micrometer drum movement and observations I 6 - 3 O 
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with counterclockwise rotation. The magnitude of the 
differences in sample standard deviations is again small - 
all are less than 0.13'* Observers 1, 3 and 5 again measure 
slightly smaller angles when using counterclockwise micro- 
meter rotation. Their average Ae'sare -0.20', -0.32* and 
-0.48', respectively. In addition, observers 2 and 4 have 
negative Ae's. Therefore, the direction of rotation of the 
micrometer drum may introduce some small systematic error 
into sextant angle measurements. 

E. INDEX ERROR DETERMINATION 

Index errors were determined by each observer every day 
before measuring angles. The index corrections for this 
project are summarized in Table 0-5* Each index correction 
was derived using the method described by Umbach (1972). 

This procedure consists of holding the sextant vertically, 
observing the sea horizon, bringing the direct and reflected 
images into coincidence and reading the micrometer and 
vernier. This is repeated several times, alternately 
turning the micrometer drum clockwise and counterclockwise . 
The results are averaged, and this average is the index 
correction. From Table C-5 it is seen that index correc- 
tions for the same sextant vary between individuals by as 
much as two minutes. Even the same observer had differences 
of up to 1.7 minutes from day to day. 



Instead of this method of daily determination of index 
corrections, the observations in Table C-^ can be averaged 
for each observer. The results of this are shown below 
in Table VII. 



Observer 


1 


2 


3 


4 


5 


6 


Sextant #2972 


- 0.50 


- 0.68 


- 0.86 


-0 .46 


- 0.50 


- 0.74 


Sextant #2982 


-0.44 


-0.26 


-0.44 


- 0.25 


- 0.32 


-0.53 


Sextant #3003 


-0.39 


- 0.38 


-0.64 


- 0.09 


- 0.50 


-0.64 


TABLE VII. Index Corrections 


from Thirty 


Observations 



(Minutes of Arc) . 



These results show differences from the index 
corrections determined by the daily method of as much as 
1.5' indicating a systematic error. If there were no 
changes in index correction during the project the averages 
in Table VII should be more accurate than the daily 
determined correctors. Applying these newly determined 
index correctors would alter the sample means of all the 
previous data, but not the sample standard deviations. 

These observations suggest that each angleman should 
determine an index correction for his sextant by averaging 
ten to fifteen measurements in each direction thereby 
minimizing the magnitude of the systematic error. 



F. CRUISE II SYSTEMATIC ERROR 

Table C-1 showed that the observations of angle 110/205 
during Cruise II had a positive bias of about 1.6'. Several 
possible error sources for this systematic error were 
considered and are described below: 

(1) Mislocation of Signals llO or 205: This did not 

occur since the angles between these stations during Cruise 
I showed no such positive bias. 

(2) Consistent Errors in Theodolite Positioning: This 

could only occur if the theodolites were mispointed on the 
initial. This is highly unlikely due to the large number 
of such pointings and the good agreement with the azimuth 
check stations. 

(3) Signals at Different Elevations: The excess angle 

measured due to differing elevations ranged from 2.5" to 
3-5" throughout the Cruise II work area. This was much 
smaller than the 1.6' bias. 

(4) Incorrect Index Corrections: It is very unlikely 

that all six sextant observers could have made the same 
large mistake. 

(5) Collimation Error: This error occurs when the 

scope is not parallel to the frame of the sextant and is 
always positive. No detectable collimation error was found 
when the sextants were examined using the method outlined 
in The Admiralty Manual of Hydrographic Surveying (I 962 ). 
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(6) Incorrect Instrument Error: This value is 

determined for 15 degree increments of every sextant by 
the manufacturer and is attached to the inside of each 
sextant case. All three sextant cases had identical instru- 
ment errors posted for each angle - 0.0'. This suggested 
that the sextants should be checked for instrument errors. 

The method suggested by CDR Wintermyre and described 
earlier on page 27 was used in an attempt to evaluate the 
instrument error of all three sextants. Thirty observations 
were made with each sextant on two different angles . As 
determined by theodolite, angle one was 2805I.22' and angle 
two was 47^03. 00 '. The differences between the theodolite 
measured angle and the sextant angle should only depend on 
theodolite error, index error and instrument error if the 
sextants are in otherwise good adjustment. The set of four 
theodolite measurements for each angle resulted in spreads 
of 03.6" and 07.8" respectively. Hence, theodolite errors 
were minimal. The mean index corrections that were deter- 
mined from 180 measurements for each sextant were - 0 . 40 ' for 
sextant #2972, - 0 . 35 ' for #2982 and - 0 . 45 ' for # 3003 * The 
index corrections arrived at by the author before using the 
sextant template were -0.66', - 0 . 34 ' and - 0 . 54 ' for the 
same three sextants. The difference between the theodolite 
angles and the mean of the 30 sextant angles (corrected for 
the author's index corrections) are shown in Table VIII. 
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Sextant 


Difference (Angle One) 


Difference (Angle Two) 


2972 


-0.72' 


- 0 . 55 ’ 


2982 


-0.65' 


-O.9O' 


3003 


-0.91 ' 


- 0 . 55 ' 



TABLE VIII. Instrument Errors 

It should be emphasized that the values in Table VIII are 
errors, not corrections. Therefore, the angles measured by 
the three sextants are 0 . 5 ' to 1 . 0 ' too small. This is 
contrary to the results from Cruise II which showed the 
sextant angles to be about 1 . 6 ' too large. 

In summary, all the possible errors have been 
considered. The 1 . 6 ' bias results from some unknown source. 

S 

Nevertheless, the standard deviation of the random errors 
from Cruise II is not affected by this error in the sample 
means . 
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V. CONCLUSIONS 



The standard deviation of the random errors in 
horizontal sextant measurements was found to vary mainly 
with the clarity of the signals being observed and the 
stability of the vessel. The differences due to observer 
experience were quite small for both the cruise data and 
the Wharf Number 2 data. Signal clarity was evaluated by 
measuring angles both with and without scopes on the sex- 
tants. This was isolated for only the Wharf Number 2 data. 
The cruise data showed the combined effects of signal 
clarity and platform stability and included some unknown 
amount of error due to theodolite positioning. 

The general range of the sample standard deviations 
from the experimental data are summarized below in Table IX. 



Platform Stability 


Scopes/No Scopes 


Sample Standard 
Deviation 


Unstable (Cruise I) 


No Scopes 


2.3'-3.6' 


Somewhat stable 
(Cruise II) 


Scopes 


1.3’-1.8' 


Very Stable 

(Wharf Number 2) 


No Scopes 


0.9'-2.0* 


Very Stable 


Scopes 


0.3’-0.8’ 



(Wharf Number 2) 

TABLE IX. Summary of Ranges of Sample Standard 



Deviations . 
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Although the above table illustrates the range of 
values for all observers, the cumulative statistics for all 
the data collected under each set of conditions give a 
better estimate of the magnitude of the random errors . The 
sample standard deviation for all angles measured with no 
scopes on an unstable platform is 2.69' and represents the 
probable upper bound. The lower limit for the sample — 
standard deviation was derived from the Wharf Number 2 data 
with scopes and is determined to be 0.64*. 

Index corrections for a given sextant may vary from 
individual to individual, but probably not by more thain 
0.5'* Some individuals measure consistently smaller or 
larger angles (by as much as 0.6’) depending on the direc- 
tion of rotation of the micrometer drum. Therefore, for 
increased accuracy, index corrections should be determined 
by averaging ten or more measurements in each direction. 

Some suggestions for further work in this area are 
appropriate. By using three T-2 theodolites to obtain a 
least square adjusted ship position, a more accurate set 
of statistics could be obtained for angles measured at sea. 
Various types of signals could be used at station locations 
to further determine the dependency of random error on 
signal clarity. The most variable quantity in attempting to 
quantify the random errors in horizontal sextant measure- 
ments is the ability of the observer. Although over 1400 
horizontal angles were analyzed, only six sextant observers 
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were used. Further work with different observers would 
give a broader data base. Nevertheless, this study does 
present analytically derived values for the standard devia- 
tion of random errors in horizontal sextant measurements 
where only estimates existed before . 



V 
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APPENDIX A: BLUNDERS AND SYSTEMATIC ERRORS 

AFFECTING THREE-POINT FIX POSITIONING ACCURACY 

The potential blunders associated with three-point fixes 
include the following: 

(1) Misread Sextant: This blunder is not readily 

identified for individual fixes. However, when conducting 
a hydrographic survey, consecutive fixes fall in a straight 
line if the vessel is carefully steered. A misread sextant 
angle will cause the fix to deviate from this line. The 
fix is then either rejected or an artificial position is 
created using dead reckoning. 

(2) Misplotted Fix: This generally occurs only with 

manually plotted fixes and is identified by the same method 
as the error above. It is corrected by simply replotting 
the fix. 

(3) Improper Identification of a Signal: This error, 

like the above two errors, is not easily discovered when 
only one fix is taken. Even along a carefully steered 
survey line, it may go undetected if the same erroneous 
signal is used throughout. But if the observer switches 
from that signal to a correct signal while on line the 
resulting fix will deviate from the straight line created 
by the previous fixes. The sextant data may be retained 
and the correct positions determined if the misidentif ied 
signal can be properly identified. 
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The systematic errors that result in reduced accuracy 
of three-point fixes are as follows: 

(1) Weak Fix Geometry: Strong geometry exists for a 

three-point fix when two of the three lines of position 
intersect at right angles. A fix has weak geometry when 
the three lines of positions approach coincidence . This 
greatly increases the effect of other errors on positional 
accuracy. Various fix geometries are discussed hy Umbach 
( 1976 ), Bowditch ( 1977 ) and Dedrick (1978). The effects 

of weak fix geometry are minimized by following the general 
rule specified by these authors. 

(2) Station Positions in Error: This error is similar 

to misidentifying a signal and its detection is also similar 
if the station position error is large. Small errors in 
station positions will often be undetected and will always 
be present in any three-point fix. Heinzen (1977) and 
Dedrick (1978) both discuss the three-point fix positional 
errors caused by incorrect station positions. 

(3) Phase Error: The apparent displacement of a 

signal due to unequal illumination of its surface is called 
phase error. It is dependent on the shape of the signal, 
the angle of the sun with the line of sight, and the 
intensity of the sunlight. Water tanks may be especially 
susceptible to this kind of error. Formulas for correction 
of phase are usually not practicable due to the numerous 
factors upon which the correction depends [Gossett 1971] • 



50 








I 4 



(4) Observer and Observed Signals Not at the Same 
Elevation; The angle observed between signals with 
elevations differing from that of the observer are called 
inclined angles. This error is minimized by choosing 
signals that are at the same elevation as the observer. If 
this is not possible the inclined angle can be reduced to a 
horizontal angle by using the following formula from 
Umbach (1976); 



/.. cos 
cos C = 



0 - sin(ht )sin(h? ) 
cos (hi) cos (h 2 ) 



C = the horizontal angle 
0 = the observed inclined angle 

hi = the angular elevation of station 1 above the 
observer 

h£ = the angular elevation of station 2 above the 
observer 






(5) Two-Observer Eccentric Error; This error is caused 
by the angle observers not being at exactly the same point. 
The magnitude is dependent on the distance between the 
observers and the angle of intersection of the two lines of 
position. Dedrick (1978) discusses this error and shows 
that for a separation distance of 3*0 feet and a 50 ° angle 
of intersection, the maximum error in the position is about 
^.3 feet. It is minimized by selecting strong fix geometry 
and by having the angle observers stand as close together 

as possible. 

(6) Horizontal Refraction; Differences in the density 
of air along a line of sight can cause bending or refraction 
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of light rays. Vertical refraction is usually larger than 
horizontal refraction due to the air "being stratified with 
denser layers near the ground. These layers are not hori- 
zontal over terrain that is sloping or unevenly heated and 
hence, horizontal refraction occurs. A line of sight pass- 
ing partly over water and partly over land is an example of 
unevenly heated terrain. Errors due to horizontal refrac- 
tion can "be as large as 10 to 18 seconds of arc [Gossett 

1971] . 

(7) Sextant Parallax*. This is caused "by the separation 
"between the center of the index mirror and the line joining 
the telescope axis and horizon glass (usually about 4 to 6 
cm). It decreases as the range to the station increases. 

For a separation of 4.3 cm the parallax correction decreases 
from 0.49' of arc at 1000 feet to 0.05' of arc at 10,000 
feet [Dedrick 1978] . 

(8) Sextant Errors: There are seven sources of error in 

the modern sounding sextant - 4 adjusta"ble and 3 nonadjust- 
a"ble [Bowditch 1977] • One nonad justa"ble error is called 
prismatic error and results from the two faces of the mirrors 
not "being parallel. The other two nonad justable errors are 
graduation errors (due to the arc or micrometer being 
improperly cut) and centering error (due to the index arm 
not pivoting at the exact center of curvature) . These are 
usually combined into one error called instrument error for 
which the manufacturer provides a correction table. The 



52 



adjustable errors are those resulting from nonperpendic- 
ularity of (1) the frame and the index mirror, and (2) the 
frame and horizon glass (side error) and the lack of 
parallelism between, (3) the index mirror and horizon glass 
at zero setting (index error), and (^) the telescope to the 
frame (collimation error). Bowditch (1977) explains each 
of these errors in detail and methods of adjustment to 
minimize them. 
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APPENDIX B; THEODOLITE INTERSECTION POSITION ERROR 



The best estimate for each sextant angle observed aboard 
ACANIA was derived from the corresponding T -2 theodolite 
determined position. Errors in these positions caused 
inaccuracies in the best estimates of the angles. The 
magnitude of these positional errors was dependent on the ' 
angular resolution of the theodolite and the distance from 
the theodolite to the position. A well-trained observer 
using a T -2 theodolite on a stationary target during day- 
light hours can measure an angle within -2.5" ninety percent 
of the time when sixteen plate settings are used [Cervarich 
1966]. This yields a standard deviation of 1 . 5 "* However, 
for only one observation on a moving target such as ACANIA, 
a larger value must be used. Heinzen ( 1977 ) states that the 
angular error in measuring azimuths for hydrographic vessel 
positioning is -36 seconds. He does not present the method 
used to derive this value, what probability is associated 
with it or to what instrument it applies. Experience 
indicates that this value is quite large, but to derive the 
largest expected error in theodolite positioning of a 
moving target it was assumed that the standard deviation of 
the T -2 theodolite measurements was 36 seconds. A more 
reasonable value of 20 seconds was also used for comparison. 
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The theodolite positioning error had varying effects on 
the computed best estimates used for the sextant angles. If 
displacement was along the circular line of position deter- 
mined by the ship and the two signals then the error in the 
best estimate was zero. Displacement normal to this line 
resulted in the maximum error. The maximum errors were 
determined for three points chosen near the extremes of the 
work area and are summarized in Tables B-1 and B-2 . The 
derivation of the values in these tables follows. 

The root mean square error (d^ms) a position is the 
square root of the sum of the squares of the standard 
deviations along the major and minor axes of a probability 



The values and Oy are not the same as the standard 

deviations of the errors in the lines of position which are 
given by and 02 * However, for two independent lines of 
position, they are related by the following two equations 
from Bowditch (1977): 



B is the angle of cut or angle of intersection of the two 
lines of position. The result of substituting these 
equations into the formula for root mean square error is 



ellipse. This is given by the equation 1 d;pnis 





[ 0 ^ + a 

2 s i n ^ B ^ 




- 4sin^Ba^0^ ] 



1 2 
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For an azimuthal line of position the standard deviation 
of the error is of the form cr i = r^ sina for small values 
of a . rj_ is the range from the point to the theodolite 
station and a is the angular resolution or the standard 
deviation of the angular measurement. By substitution the 
root mean square error for azimuthal systems is finally 

. The probability 



given by 1 drms = 
associated with root mean square error is not constant but 
varies with the relationship between 

probabilities in Table B-1 that result from the three values 
of Oy/ o^ were derived from Bowditch (1977). 

The errors in the best estimates of the angles (01, 

0 2 and 0^) in Table B-2 were derived by contouring the 
errors which resulted from shifting the theodolite deter- 
mined positions + 1 meter in latitude and longitude. An 

example is shown below in Figure B-1 . 

-1' O' +1' 

18' 1 + 1 .35' 

X 




h-1.53’ 



+1 .?t ■ 

X 



Figure B-1 . Angular Error at Position 1 Due to 2 d;pnis 

Theodolite Positioning Error of 0.59^ Meters 
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POSITION 



1 



2 



3 



Latitude (N) 
Longitude (W) 



36°37'09-825" 
121°53'30 .088" 



36036 ' 48 . 222 " 
12l°53'03.284" 



36°37'07.210 

l2l°52'3l .133 





1064 m 


1933m 


2520m 


rz 


1171m 


715m 


1701 m 


6 


68.6° 


72 . 4 ° 


47.1° 


ai 


20" 


20" 


20" 




.103m 


.187m 


. 244 m 


^^2 


. 114 m 


. 069m 


.165m 




• 137m 


.198m 


. 37 ^m 




.092m 


.069m 


. 147m 


V^x 


.674 


.348 


.393 


^ ‘^rms 


. 165m 


. 2l0m 


.40 2m 


Prob. (1 dj^mg) 


64 . 3 ^ 


67. 4 ?^ 


67-2% 


^ *^rms 


.330m 


. 420 m 


. 804 m 


Prob. (2 dj^jjjg) 


97.6?$ 


96.3% 


96 '. 5 ^ 


^^2 


36" 


36" 


36" 




. 186 m 


.337m 


. 440 m 


^2 


. 204 m 


.125m 


.297m 




. 246 m 


.358m 


.675m 




. I66m 


. 124 m 


. 264 m 


^y/^x 


.674 


.3^9 


.392 


^ ^rms 


.297m 


.377m 


.725m 


Prob. (1 dj.mg) 


64 . 3 ?^ 


67. 4 ?^ 


67 >2% 


2 <^rms 


. 594 m 


. 75 ^m 


1 .450m 


Prob. (2 d]~ins) 


97 - 6 fo 


96.3% 


96.5^5 



If 

ft 



TABLE B- 1 . Theodolite Positioning Errors at Three 
Locations 
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POSITION 



1 



2 



3 



Latitude (N) 
Longitude (W) 



36°37’09.825" 

121 ° 53 ’ 30 . 088 " 



36036*48 . 222 " 
121053'03 .284" 



36037*07 .210" 
121°52 *31 .133" 





20" 


2 dj-nis 


0.330m 


error 


0.5’ 


02 error 


1.5’ 


0^ error 


1.7* 


“2 


36" 


^ ^rms 


0 .594m 


error 


0.9’ 


02 error 


2.7* 


0^ error 


3.0* 



20" 


20" 


0 . 420 m 


0 . 804 m 


1 .9* 


1 .2* 


2.1* 


1 .2* 


2.2* 


1 . 3 ’ 


36" 


36" 


0 . 754 m 


1.450m 


3.5’ 


2.1* 


3.8* 


2.1 * 


3.9’ 


2.3* 



TABLE B-2 . Maximum Errors in Angular Best Estimates at 

Three Locations Due to Theodolite Positioning 
Errors . 
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The origin in the above figure represents position 1 . 

The grid around the origin are 1 meter shifts in latitude 
and longitude. The value at each grid point is the differ- 
ence in the angle 0J at that point compared to the origin. 

By drawing the appropriate d^ms circle (in this case 2 d;r.nis 
= 0.59^ni) the corresponding angular error can be found. 

It can be seen from Table B-2 that the angular errors 
induced by the theodolite positioning errors are not 
insignificant. However, the precise amount of error in the 
best estimates is indeterminant since the angular resolution 
of the T-2 theodolite angles is not known. The average 
distance between the theodolite determined positions and 
the corresponding mean sextant fixes for both cruises was 
1.05m. In conclusion, the random errors in the horizontal 
sextant angles measured aboard ACANIA were made larger by 
the errors due to theodolite positioning. Hence, the values 
of standard deviation for the cruise data are the maximum 
expected errors for sextant angles measured at sea. 
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APPENDIX C: DATA SET STATISTICS 



The coding system used for the data sets is shovm 
below. All values shown are in minutes of arc. 

(1) Data Origin: 

I - Cruise I 

II - Cruise II 

W - Wharf Number 2 
B - Beach Lab 

(2) Use of Scopes on Sextants: 

N - no scopes 

S - with scopes 

(3) Observer Number (experience level decreases as 

this number increases): 

1 through 6 

(4) Angle Designator: 

0 1 - 110/205 

©2 - 205/202 
63 - 205/107 

04 - 206/110 

05 - 110/302 

(5) Repeated Data Set (if necessary): 

( 1 ) - first set 

( 2 ) - second set 
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TOTAL 


INI - 0 1 


IN2-01 


IN 3-0 1 


IN 4 - 0 1 


IN 5 - 0 1 


IN6-01 


(32) 












e 


+0.09 


-0.83 


-1.16 


+0.36 


-0 .44 


+6.37 


/S 

a 


2.25 


2.63 


2.62 


3.33 


2.34 


3.59 


TOTAL 


IN 1 - 0 i( 2 ) 


IN 2-02 


IN 3-®3 








( 10 ) 












e 

/S 


+ 0 . 08 


+ 0 . 88 


+ 1 .26 








a 


3.84 


2.55 


3.44 








TOTAL 

(32) 

e 


IISl -01 


IIS 2-01 


IIS 3 -©i 


IIS 4 -©i 


IIS 5 -®! 


IIS 6 -©i 


+0.93 


+ 1.31 


+ 1 .85 


+ 1 .78 


+ 0 .88 


+2 . 1 8 


/N 

a 


1 .28 


1.76 


1.25 


1.61 


1.67 


1.28 


TOTAL 

(32) 

i 


IISl- 0 i( 2 ) 


IIS 2-02 


IIS 3-©3 


IIS 4-03 


IIS 5 -® 1 ( 2 ) 


IIS 6-02 


+ 2.40 


- 0.24 


+ 1 .48 


-0.45 


+ 1 .74 


-0.73 


a 


1.58 


1 .41 


1.67 


1.77 


1.58 


1.05 



TABLE C- 1 . Cruise I and Cruise II Data. 
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CLOCKWISE 

MICROMETER ( 15 ) WNl -04 WN 2-04 WN 3-04 WN ^-04 WN 5-04 WN 6-04 



e 




+2.88 


+0.86 


-0.26 


-3.38 


+0.63 


+0.08 


a 




1 .25 


0.92 


1.12 


1.15 


0.47 


0.80 


COUNTERCLOCKWISE 
MICROMETER ( 15 ) 

i 


+ 2.54 


+1 .88 


-0.94 


- 2.64 


+0 . 20 


-1 .26 


a 




0.96 


0.78 


1 .38 


0.64 


1 .17 


0.82 


TOTAL ( 30 ) 


e 




+2.71 


+1.37 


-0.60 


-3.01 


+ 0.42 


-0.59 


0 




1.11 


0.98 


1.28 


0.99 


0.90 


1.05 


Ai 




-0.34 


+ 1 .02 


-0.68 


+0.74 


-0.43 


-1.34 


CLOCKWISE 


MICROMETER 


( 15 ) 


WNl-05 


WN2-03 


0 

1 


WN 4-05 


WN5-05 WN6-05 


e 




+ 4 ,. 43 


+1.79 


+1.77 


+2.35 


-0.51 


+0.92 


a 




0.85 


1 .10 


1.10 


0.96 


1 .12 


0.74 


COUNTERCLOCKWISE 
MICROMETER ( 15 ) 

e 


+2.62 


+3.04 


+ 0.48 


+2 .04 


-0.90 


+2 .78 


a 




1.15 


0.71 


0.69 


0.91 


0.48 


1 .06 


TOTAL ( 30 ) 


e 




+3.52 


+ 2.42 


+1.13 


+ 2.19 


-0.70 


+ 1 . 85 


a 




1.35 


1.11 


. 1.12 


0.93 


0.87 


1 .31 


Ae 




- 1 .81 


+1 .25 


-1 .29 


-0.31 


- 0.40 


-hi . 86 


TABLE 


C- 2 . 


Wharf 


Number 


2 Data 


- Without Scopes. 
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CLOCKWISE 

MICROMETER (15) WSl-©4 WS2-04 WS3-04 WS4-04 WS5-04 WS6-04 



e 


+ 1 .04 


+ 0 . 58 


-0.43 


+ 0.10 


+0.83 


+0.44 


a 


0.25 


0.23 


0.33 


0.51 


0.51 


0.48 


COUNTERCLOCKWISE 
MICROIVIETER (15) 














£ 


+0 . 66 


+0.41 


-0.62 


-1.16 


+0 .14 


-0.03 


a 


0.27 


0.21 


0.32 


0.44 


0.31 


0.57 


TOTAL (30) 














e 


+0.85 


+0.50 


-0.52 


-0.53 


+0.49 


+0.21 


g 


0.32 


0.23 


0.33 


0.79 


0.55 


0.57 


Ae 


-0.38 


-0.17 


-0.19 


-1.26 


-0.69 


-0.47 


CLOCKWISE 
MICROMETER (15) 


WSl-05 


WS2-0 5 


WS3-® 5 


WS4-05 


WS5-05 


WS6-0 


5 


+0 . 62 


-0.37 


+0.33 


+0 .04 


+0.10 


+0 . 66 


a 


0.18 


0.24 


. 0.14 


0.42 


0.28 


0.46 


COUNTERCLOCKWISE 
MICROMETER (15) 














g 


+0.44 


+0.03 


+0.10 


-0.25 


-0.36 


+1 .34 


a 


0.30 


0.32 


0.25 


0.23 


0.22 


0.45 


TOTAL (30) 














i 


+0.53 


-0.17 


+0.21 


-0.10 


-0.13 


+1 .00 


a 


0.26 


0.35 


0.23 


0.36 


0.34 


0.57 


Ae 


-0.18 


+0.40 


-0.23 


-0.29 


-0.46 


+0 . 67 



TABLE C-3* Wharf Number 2 Data - With Scopes 
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I 




41 



IF 







CLOCKWISE 

MICROBffiTER 


( 15 ) 


BSl(l) 


BS 2 ( 1 ) 


SEXTANT #2972 
BS 3 ( 1 ) BS 4 ( 1 ) BS 5 ( 1 ) 


BS 6 ( 1 ) 


i 




-0 .41 


-0.59 


-0.77 


-0.35 -0.28 


-0.91 


✓N 

a 




0.28 


0.25 


0.21 


0.25 0.24 


0 .44 


COUNTERCLOCKWISE 
MICROMETER ( 15 ) 

e 


-0.59 


-0.77 


-0.94 


-0.56 -0.71 


-0.58 


/V 

a 




0.33 


0.14 


0.14 


0.34 0.30 


0.42 


TOTAL (30) 


e 




-0.50 


- 0.68 


- 0.85 


- 0.46 -0.50 


-0.75 


/s 

g 




0.32 


0.21 


0.20 


0.31 0.34 


0.45 


Ae 




- 0.18 


- 0.18 


-0.17 


-0.21 -0.43 


+0.33 


CLOCKWISE 

MICROMETER 


( 15 ) 


BS 1 ( 2 ) 


BS 2 ( 2 ) 


SEXTANT #2982 
BS 3 ( 2 ) BS 4 ( 2 ) BS 5 ( 2 ) 


BS 6 ( 2 ) 


i 




- 0.28 


- 0.04 


-0.25 


+0.09 -0.09 


-0.67 


a 




0.36 


0.23 


0.28 


0.36 0.28 


0.45 


COUNTERCLOCKWISE 
MICROMETER ( 15 ) 

£ 


-0.59 


-0.47 


-0.63 


-0.59 -0.55 


-0.39 


/S 

a 




0.34 


0.24 


0.16 


0.35 0.25 


0.34 


TOTAL (30) 


e 




-0.43 


-0.25 


- 0.44 


-0.25 -0.32 


-0.53 


a 




0.38 


0.32 


0.30 


0.49 0.35 


0.42 


Ae 




-0.31 


+ 0.01 


-0.38 


- 0.68 - 0.46 


+ 0.24 



TABLE C- 4 . Index Correction Differences 
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CLOCKWISE SEXTANT #3003 

MICROMETER ( 15 ) BS 1 ( 3 ) BS 2 ( 3 ) BS 3 ( 3 ) BS 4 ( 3 ) BS 5 ( 3 ) BS 6 ( 3 ) 



i 


- 0.33 


-0.19 


-0.43 


+0.39 


-0 .23 


-0.63 


a 


0.31 


0.20 


0.23 


0.39 


0.30 


0.63 


COUNTERCLOCKWISE 
MICROiyiETER ( 15 ) 


i 


-0 .45 


-0.57 


-0.85 


-0.57 


-0.78 


- 0.64 


a 


0.37 


0.22 


0.22 


0.30 


0.40 


0.50 


TOTAL ( 30 ) 


i 


-0.39 


-0.38 


- 0.64 


-0 .09 


-0.50 


-0.63 


a 


0.34 


0.28 


0.31 


0.60 


0.45 


0.58 


Ae 


-0.12 


-0.38 


- 0.41 


-0.96 


-0.55 

• 


- 0.01 



TABLE C-4. Index Correction Differences (continued) . 
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OBSERVER 


1 


2 


3 


4 


5 


6 


WITH SCOPES 














Sextant 

#2972 


— 


• • • 

000 
1 1 1 


+0.1 


+ 1 .0 




-1.0 

-1.0 

-0.6 

-0.5 


#2982 


-0.6 


-0.5 


1111 

0 0 0 


-0.5 

-0.6 

-0.2 


— 


— 


#3003 


-0.6 

-0.5 

-0.5 

-0.4 

-0.7 


0.0 


■ 


+ 0 .4 


0.0 

-0.5 

-0.6 

-0.5 

-0.5 


+0.3 


NO SCOPES 














Sextant 














#2972 


+0 . 5 


1 1 
0 0 


+0.5 
-1 .2 


-0.2 
-1 .0 




1 1 
0 i-* 


#2982 


— ^ 


-0.4 

-1.0 


1 1 1 

0 

0 00 0 


-1.0 


+0 . 6 


+0 . 8 
+1 . 0 


#3003 


-0.5 

-0.5 

-0.4 

-0.6 






+0 .4 


-0.3 
-1 .0 
-0.5 
-0.6 


+0 . 2 



-0.4 



TABLE C-5* Abstract of Index Correctors From 
Cruise and Wharf Number 2 Data. 
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